Associative memories, also referred to as content addressable memories, are widely used in the fields of pattern matching and identification, expert systems and artificial intelligence. A widely used associative memory is the Hopfield artificial neural network. Hopfield artificial neural networks are described, for example, in U.S. Pat. No. 4,660,166 to Hopfield entitled Electronic Network for Collective Decision Based on Large Number of Connections Between Signals.
Unfortunately, there is a fundamental scaling problem that can limit the use of associative memories to solve real-world problems. In particular, associative memories generally provide an N2 or geometric scaling as a function of inputs. This geometric scaling may be unreasonable to support applications at the scale of complexity that warrants such technology.
Associative memories are also described in U.S. Pat. No. 6,581,049 to coinventor Aparicio, IV et al., entitled Artificial Neurons Including Power Series of Weights and Counts That Represent Prior and Next Association, assigned to the assignee of the present application, the disclosure of which is hereby incorporated herein by reference in its entirety as if set forth fully herein. As described in the Abstract of the '049 patent, an artificial neuron includes inputs and dendrites, a respective one of which is associated with a respective one of the inputs. Each dendrite includes a power series of weights, and each weight in a power series includes an associated count for the associated power. The power series of weights preferably is a base-two power series of weights, each weight in the base-two power series including an associated count that represents a bit position. The counts for the associated power preferably are statistical counts. More particularly, the dendrites preferably are sequentially ordered, and the power series of weights preferably includes a pair of first and second power series of weights. Each weight in the first power series includes a first count that is a function of associations of prior dendrites, and each weight of the second power series includes a second count that is a function of associations of next dendrites. More preferably, a first and second power series of weights is provided for each of multiple observation phases. In order to propagate an input signal into the artificial neuron, a trace preferably also is provided that is responsive to an input signal at the associated input. The trace preferably includes a first trace count that is a function of associations of the input signal at prior dendrites, and a second trace count that is a function of associations of the input signal at next dendrites. The first and second power series are responsive to the respective first and second trace counts. The input signal preferably is converted into the first and second trace counts, and a trace wave propagator propagates the respective first and second trace counts into the respective first and second power series of weights.
Published U.S. patent application 2003/0033265 to coinventor Cabana et al. entitled Artificial Neurons Including Weights That Include Maximal Projections, the disclosure of which is hereby incorporated herein by reference in its entirety as if set forth fully herein, can allow lossless compression without requiring geometric scaling. In particular, as noted in the Abstract of this published patent application, an artificial neuron includes inputs and dendrites, a respective one of which is associated with a respective one of the inputs. A respective dendrite includes a respective power series of weights. The weights in a given power of the power series represent a maximal projection. A respective power also may include at least one switch, to identify holes in the projections. By providing maximal projections, linear scaling may be provided for the maximal projections, and quasi-linear scaling may be provided for the artificial neuron, while allowing a lossless compression of the associations. Accordingly, hetero-associative and/or auto-associative recall may be accommodated for large numbers of inputs, without requiring geometric scaling as a function of input.
One conventional use of correlational matrices, which may be similar to associative memories, is in spatial representation and prediction. Spatial representation and prediction can apply to many different fields across many scientific disciplines. As an example in applied engineering, spatial prediction may be used in geostatistics to predict unknown values given a set of known values across some continuous map. As an example in pure science, there is a long history in psychology and neurology about the representation of “cognitive maps”, perhaps an associative memory of spatial objects used for foraging and wayfinding. There is also extensive literature on machine-based pattern recognition, often applied to optical character and handwriting recognition.
Spatial prediction in geostatistics may incorporate some measure of spatial dependence. However, the standard variogram and Kriging methods are usually applied to prediction of a single contiguous variable (for example, using SAS and/or other standard statistical packages). Assuming continuity of values, co-variance is a function of distance. Given the data values at several points in a map, such methods predict values for the same variable at other nearby points in the map, using some form of interpolation and/or extrapolation.
In biological systems, neural designs of realistic neural theories are emerging, such as William Calvin's Cerebral Code. Calvin's analysis of neural recruitment forms triangular structures of fixed distances within pre-wired grid spaces.
Machine-based pattern recognition may address image patterns (bit patterns) per se. Given patterns of bits, such methods may work to classify a pattern as a known type (such as a particular letter) and/or to complete the pattern within a well-structured grid (such as occluded bits of a letter grid).